Papers with mid-training model
How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders (2025.findings-emnlp)
Copied to clipboard
Tatsuro Inaba, Go Kamoda, Kentaro Inui, Masaru Isonuma, Yusuke Miyao, Yohei Oseki, Yu Takagi, Benjamin Heinzerling
| Challenge: | Using sparse autoencoders, we explore how bilingual language models develop complex internal representations. |
| Approach: | They employ sparse autoencoders to analyze bilingual language models' internal representations. |
| Outcome: | The proposed method integrates decomposed representations from a fully trained model into a mid-training model. |